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Abstract 

We characterize the capacity region to within 2 bits/s/Hz and the symmetric capacity to within 
Q , 1 bit/s/Hz for the two-user Gaussian interference channel (IC) with feedback. We develop achievable 

schemes and derive a new outer bound to arrive at this conclusion. One consequence of the result is 

^ ■ that feedback provides multiplicative gain, i.e., the gain becomes arbitrarily large for certain channel 

00 ' 

parameters. It is a surprising result because feedback has been so far known to provide no gain in 



memoryless point-to-point channels and only bounded additive gain in multiple access channels. The gain 



m 
en 

ly^ , comes from using feedback to maximize resource utilization, thereby enabling more efficient resource 

sharing between the interfering users. The result makes use of a deterministic model to provide insights 



into the Gaussian channel. This deterministic model is a special case of El Gamal-Costa deterministic 
model and as a side-generalization, we establish the exact feedback capacity region of this general class 
of deterministic ICs. 

Index Terms 

Feedback Capacity, The Gaussian Interference Channel, A Deterministic Model 

I. Introduction 

Shannon showed that feedback does not increase capacity in memoryless point-to-point chan- 
nels f\\. On the other hand, feedback can indeed increase capacity in memory channels such as 
colored Gaussian noise channels. While it can provide multiplicative gain especially in the low 
SNR regime, the gain is bounded, i.e., feedback can provide a capacity increase of at most one 

May 20, 2010 DRAFT 



bit 0, [El. In the multiple access channel (MAC), however, Gaarder and Wolf (41] showed that 
feedback could increase capacity although the channel is memoryless. Inspired by this result, 
Ozarow [5J found the feedback capacity region for the two-user Gaussian MAC. Ozarow's result 
reveals that feedback provides only additive power gain. The reason for the bounded power gain 
is that in the MAC, transmitters cooperation induced by feedback can at most boost signal power 
via aligning signal directions. Boosting signal power provides a capacity increase of a constant 
number of bits. 

A question arises: will feedback help significantly in other channels where a receiver wants 
to decode only desired message in the presence of interference? To answer this question, we 
focus on the simple two-user Gaussian interference channel (IC) where each receiver wants 
to decode the message only from its corresponding transmitter. We first make progress on the 
symmetric capacity. Gaining insights from a deterministic model [6J and Alamouti's scheme [|7]], 
we develop a simple two-staged achievable scheme. We then derive a new outer bound to show 
that the proposed scheme achieves the symmetric capacity to within one bit for all values of the 
channel parameters. 

An interesting consequence of this result is that feedback can provide multiplicative gain in 
interference channels. This can be shown from the generalized degrees-of-freedom (g.d.o.f.) in 
Fig. [H The notion was defined in [8J as 

^ ^ SNR,INR^oo logSNR 

where a (x-axis) indicates the ratio of INR to SNR in dB scale: a = /^^ ' ^, ^ . Notice that in 

^ ^ log SNR 

certain weak interference regimes (0 < a < |) and in the very strong interference regime 
{a > 2), feedback gain becomes arbitrarily large as SNR and INR go to infinity. For instance, 
when a = i, the gap between the non-feedback and the feedback capacity becomes unbounded 
with the increase of SNR and INR, i.e.. 



1 
4 



C^sym - C^sym ^ ^ log SNR ^ OO. (2) 



Observing the ratio of the feedback to the non-feedback capacity in the high SNR regime, one 
can see that feedback provides multiplicative gain (50% gain for a = ^): -^^ -^ 1.5. 

Moreover, we generalize the result to characterize the feedback capacity region to within 2 
bits per user for all values of the channel parameters. Unlike the symmetric case, we develop an 
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Fig. 1. The generalized degrees-of- freedom of the Gaussian interference channel (IC) with feedback. For certain weak 
interference regimes (0 < a < |) and for the very strong interference regime (a > 2), the gap between the non-feedback 
and the feedback capacity becomes arbitrarily large as SNR and INR go to infinity. This implies that feedback can provide 
unbounded gain. 



infinite- Staged achievable scheme that employs three techniques: (i) block Markov encoding [0, 
ifTOll : (ii) backward decoding [11 J; and (iii) Han-Kobayashi message splitting [[1211 . This result 
shows interesting contrast with the non-feedback capacity result. In the non-feedback case, it 
has been shown that the capacity region is described by five types of inequalities including the 
bounds for Ri + 2i?2 and 2Ri + i?2 [IT2ll . ^. On the other hand, our result shows that the 
feedback capacity region requires only three types of inequalities without the Ri + 2R2 and 
2i?i + i?2 bounds. 

We also develop new interpretation, what we call a resource hole interpretation, to provide 
qualitative insights as to where feedback gain comes from. We find that the gain comes from 
using feedback to maximize resource utilization, thereby enabling more efficient resource sharing 
between the interfering users. Also the efficient resource utilization due to feedback turns out to 
deactivate the 2Ri + R2 bound. 

Our results make use of a deterministic model ^ to provide insights into the Gaussian 
channel. This deterministic model is a special case of El Gamal-Costa model [fTSl]. As a side- 
generalization, we establish the exact feedback capacity region of this general class of determin- 
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istic ICs. From this result, one can infer an approximate feedback capacity region of two-user 
Gaussian MIMO ICs, as Teletar and Tse [fT4l| did in the non-feedback case. 

Interference channels with feedback have received previous attention [HSll . lfT6ll . ifTTll . ifTSll . 
Kramer ifTSll . lfT6ll developed a feedback strategy in the Gaussian IC; Kramer and Gastpar IfTTll 
derived an outer bound. However, the gap between the inner and outer bounds becomes arbitrarily 
large with the increase of SNR and \NRu Jiang-Xin-Garg [[18 J found an achievable region in 
the discrete memoryless IC with feedback, based on block Markov encoding [9J and binning. 
However, their scheme involves three auxiliary random variables and therefore requires further 
optimization. Also no outer bounds are provided. We propose explicit achievable schemes and 
derive a new tighter outer bound to characterize the capacity region to within 2 bits and 
the symmetric capacity to within 1 bit universally. Subsequent to our work, Prabhakaran and 
Viswanath [19J have found an interesting connection between our feedback problem and the 
conferencing encoder problem. Making such a connection, they have independently characterized 
the sum feedback capacity to within 19 bits/s/Hz. 

II. Model 

Fig. [2] describes the two-user Gaussian IC with feedback. Without loss of generality, we 
normalize signal power and noise power to 1, i.e., Pk = I, Zj. ^ CJV{0, 1), V/c = 1, 2. Hence, 
the signal-to-noise ratio and the interference-to-noise ratio can be defined to capture channel 
gains: 

SNRi^l^nr, SNR2^|^22r, 

(3) 

iNRi2 = i^i2r, iNR2i = i^2ir. 

There are two independent and uniformly distributed sources, Wk G {1, 2, • • • , rrik} , V/c = 1, 2. 
Due to feedback, the encoded signal Xki of user k at time z is a function of its own message 
and past output sequences: 

Xki = fi {Wk. ni, ■ • ■ , n(i-i)) = fk {Wk. Yi-') , (4) 

where we use shorthand notation Yl~^ to indicate the sequence up to i — 1. A rate pair (i?i, R2) 
is achievable if there exists a family of codebook pairs with codewords (satisfying power 

^Although this strategy can be arbitrarily far from optimality, a careful analysis reveals that it can also provide multiplicative 
feedback gain. See Fig. [13] for this. 
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Fig. 2. The Gaussian interference channel (IC) with feedback 

constraints) and decoding functions such that the average decoding error probabihties go to 
zero as block length A^ goes to infinity. The capacity region C is the closure of the set of the 
achievable rate pairs. 



III. Symmetric Capacity to v^ithin One Bit 
We start with a symmetric channel setting where \gii\ = \g22\ = \gd\ and \gi2\ 

SNR = SNRi = SNR2, INR = INR12 = INR21. 



mi\ 



\9c\' 
(5) 



Not only is this symmetric case simple, it also provides the key ingredients to both the achievable 
scheme and outer bound needed for the characterization of the capacity region. Furthermore, this 
case provides enough qualitative insights as to where feedback gain comes from. Hence, we first 
focus on the symmetric channel. Keep in mind however that our proposed scheme for a symmetric 
rate is different from that for a rate region: in the symmetric case, the scheme employs only 
two stages (or blocks), while an infinite number of stages are used in the general case. We will 
address this in Section HVl 

The symmetric capacity is defined by 



aym=SUp{i?:(i?,i?)GC}, 



(6) 
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SNR (dB) 



Fig. 3. The gap between our inner and upper bounds. The gap is upper-bounded by exactly one bit. The worst-case gap occurs 
when SNR ^ I N R and these values go to infinity. In the strong interference regime, the gap vanishes with the increase of S N R 
and INR, while in the weak interference regime, the gap does not, e.g., the gap is around 0.5 bits for a = ^. 



where C is the capacity region. 

Theorem 1: We can achieve a symmetric rate of 



R 



1 



1 



,sym - max I - log (1 + INR) , - log 



(l + SNR+INR)^ 



SNR 
1+INR 



1 + 2INR 



The symmetric capacity is upper-bounded by 
1 



^sym — 7: sup 



log (l + ^^\^^'^^^2^|^p ) + log (1 + SNR + INR + 2pVSNR-INR) 



(7) 



2 o<p<i 
For all channel parameters of SNR and INR, we can achieve all rates R up to Csym — 1, i.e., 



. (8) 



^sym ~ 1 ^ ^ ^ ^sym- 



(9) 



A. Deterministic Model 

As a stepping stone towards the Gaussian IC, we use an intermediate model: a linear deter- 
ministic model [6J, illustrated in Fig. |4l This model is useful in the non-feedback Gaussian IC: 
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Fig. 4. The deterministic IC with feedback 

it was shown in [l20l that the deterministic IC can approximate the Gaussian IC to within a 
constant number of bits irrespective of the channel parameter values. Our approach is to first 
develop insights from this model and then translate them to the Gaussian channel. 

The connection with the Gaussian channel is as follows. The deterministic IC is characterized 
by four values: nn, 7112,^21 and 7^22 where riij indicates the number of signal bit levels (or 
resource levels) from transmitter i to receiver j. These values correspond to channel gains in dB 
scale, i.e., Vi 7^ j. 



rii. 



LlogSNRj, n,, = LloglNR,,J. 



(10) 



In the symmetric channel, n = nn = 7122 and m = n^ = ^21. Signal bit levels at a receiver 
can be mapped to the binary streams of a signal above the noise level. Upper signal levels 
correspond to most significant bits and lower signal levels correspond to least significant bits. A 
signal bit level observed by both the receivers above the noise level is broadcasted. If multiple 
signal levels arrive at the same signal level at a receiver, we assume a modulo-2-addition. 

B. Achievable Scheme for the Deterministic IC 

Strong Interference Regime (m > n): We explain the scheme through the simple example of 
a := ^ = 3, illustrated in Fig. El Note that each receiver can see only one signal level from its 
corresponding transmitter. Therefore, in the non-feedback case, each transmitter can send only 
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Fig. 5. An achievable scheme for the deterministic IC: strong interference regime a :— — = 3. 

1 bit through the top signal level. However, feedback can create a better alternative path, e.g., 
[transmitter! -^ receiver2 -^ feedback -^ transmitter2 -^ receiverl]. This alternative path 
enables to increase the non-feedback rate. 

The feedback scheme consists of two stages. In the first stage, transmitters 1 and 2 send inde- 
pendent binary symbols {ai,a2,as) and (61,62,^3), respectively. Each receiver defers decoding 
to the second stage. In the second stage, using feedback, each transmitter decodes information 
of the other user: transmitters 1 and 2 decode (61, 62,^3) and (ai,a2,a3), respectively. Each 
transmitter then sends the other user's information. Each receiver gathers the received bits sent 
during the two stages: the six linearly independent equations containing the six unknown symbols. 
As a result, each receiver can solve the linear equations to decode its desired bits. Notice that 
the second stage was used for refining all the bits sent previously, without sending additional 
information. Therefore, the symmetric rate is | in this example. Notice the 50% improvement 
from the non-feedback rate of 1. We can easily extend the scheme to arbitrary (n,m). In the 
first stage, each transmitter sends m bits using all the signal levels. Using two stages, these m 
bits can be decoded with the help of feedback. Thus, we can achieve: 



^sym — ^ • 



(11) 



Remark 1: The gain in the strong interference regime comes from the fact that feedback 
provides a better alternative path through the two cross links. The cross links relay the other user's 
information through feedback. We can also explain this gain using a resource hole interpretation. 
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Notice that in the non-feedback case, each transmitter can send only 1 bit through the top level 
and therefore there is a resource hole (in the second level) at each receiver. However, with 
feedback, all of the resource levels at the two receivers can be filled up. Feedback maximizes 
resource utilization by providing a better alternative path. This concept coincides with correlation 
routing in [|15l1 . 

On the other hand, in the weak interference regime, there is no better alternative path, since 
the cross links are weaker than the direct links. Nevertheless, it turns out that feedback gain can 
also be obtained in this regime. 

Weak Interference Regime (m < n): Let us start by examining the scheme in the non- 
feedback case. Unlike the strong interference regime, only part of information is visible to 
the other receiver in the weak interference regime. Hence, information can be split into two 
parts pl2l| : common m bits (visible to the other receiver) and private (n — m) bits (invisible to 
the other receiver). Notice that using common levels causes interference to the other receiver. 
Sending 1 bit through a common level consumes a total of 2 levels at the two receivers (say $2), 
while using a private level costs only $1. Because of this, a reasonable achievable scheme is to 
follow the two steps sequentially: (i) sending all of the cheap (n — m) private bits on the lower 
levels; (ii) sending some number of common bits on the upper levels. The number of common 
bits is decided depending on m and n. 

Consider the simple example of a = ^ = |, illustrated in Fig. [6l (a). First transmitters 1 and 
2 use the cheap private signal levels, respectively. Once the bottom levels are used, however 
using the top levels is precluded due to a conflict with the private bits already sent, thus each 
transmitter can send only one bit. 

Observe the two resource holes on the top levels at the two receivers. We find that feedback 
helps fill up all of these resource holes to improve the performance. The scheme uses two stages. 
As for the private levels, the same procedure is applied as that in the non-feedback case. How to 
use the common levels is key to the scheme. In the first stage, transmitters 1 and 2 send private 
bits a2 and 62 on the bottom levels, respectively. Now transmitter 1 squeezes one more bit ai 
on its top level. While ai is received cleanly at receiver 1, it causes interference at receiver 2. 
Feedback can however resolve this conflict. In the second stage, with feedback transmitter 2 can 
decode the common bit ai of the other user. As for the bottom levels, transmitters 1 and 2 send 
new private bits a^ and 63, respectively. The idea now is that transmitter 2 sends the other user's 
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Delay 



Fig. 6. Achievable schemes for the weak interference regime, e.g., oi — ^ — \. 

common bit ai on its top level. This transmission allows receiver 2 to refine the corrupted bit 
62 from 62 ® cii without causing interference to receiver 1, since receiver 1 already had the side 
information of ai from the previous broadcasting. We paid $2 for the earlier transmission of ai, 
but now we can get a rebate of $1. Similarly, with feedback, transmitter 2 can squeeze one more 
bit hi on its top level without causing interference. Therefore, we can achieve the symmetric 
rate of | in this example, i.e., the 50% improvement from the non-feedback rate of 1. 

This scheme can be easily generalized to arbitrary (n,m). In the first stage, each transmitter 
sends m bits on the upper levels and (n — m) bits on the lower levels. In the second stage, each 
transmitter forwards the m bits of the other user on the upper levels and sends new (n — m) 
private bits on the lower levels. Then, each receiver can decode all of the n bits sent in the first 
stage and new (n — m) private bits sent in the second stage. Therefore, we can achieve: 



R 



n+ {n — m) 



sym 



m 



(12) 



Remark 2 (A Resource Hole Interpretation): Observe that all the resource levels are fully 
packed after applying the feedback scheme. Thus, feedback maximizes resource utilization to 
improve the performance significantly. 

Remark 3 (Exploiting Side Information): Another interpretation can be made to explain this 
gain. Recall that in the non-feedback case, the broadcast nature of the wireless medium precludes 
us from using the top level for one user when we are already using the bottom level for the other 
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user. In contrast, if feedback is allowed, the top level can be used to improve the non-feedback 
rate. Suppose that transmitters 1 and 2 send ai and hi through their top levels, respectively. 
Receivers 1 and 2 then get the clean bits ai and 6i, respectively. With feedback, in the second 
stage, these bits (ai,6i) can be exploited as side information to refine the corrupted bits. For 
example, with feedback transmitter 1 decodes the other user's bit hi and forwards it through 
the top level. This transmission allows receiver 1 to refine the corrupted bit a2 from a2 © hi 
without causing interference to receiver 2, since receiver 2 already had the side information of 
hi from the previous broadcasting. We exploited the side information with the help of feedback 
to refine the corrupted bit without causing interference. The exploitation of side information was 
also observed and pointed out in network coding examples such as the butterfly network [l2T]l . 
two-way relay channels [i22i] . general wireless networks ll23]l . and broadcast erasure channels 
with feedback [24J. 

C Optimality of the Achievable Scheme for the Deterministic IC 

Now a natural question arises: is the scheme optimal? In this section, using the resource hole 
interpretation, we provide a positive conjecture on the optimality. Later in Section [Vl we will 
provide a rigorous proof to settle this conjecture. 

From W to V Curve: Fig. [7] shows (i) the symmetric feedback rate (fTTI) . ([T2h of the achievable 
scheme (representing the "V" curve); (ii) the non-feedback capacity EOll (representing the "W" 
curve). Using the resource hole interpretation, we will provide intuition as to how we can go 
from the W curve to the V curve with feedback. 

Observe that the total number of resource levels and transmission cost depend on (n,m). 
Specifically, suppose that the two senders employ the same transmission strategy to achieve the 
symmetric rate: using x private and y common levels. We then get: 

# of resource levels at each receiver = max(n, m), 

(13) 
transmission cost =lxx + 2xy. 

Here notice that using a private level costs 1 level, while using a common level costs 2 levels. 
Now observe that as a = ^ grows: for < a < 1, transmission cost increases; for a > 1, the 
number of resource levels increases. Since all the resource levels are fully utilized with feedback, 
this observation implies that with feedback a total number of transmission bits must decrease 
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feedback 
-■■ no feedback 




Fig. 7. Symmetric feedback rate (fTTT) . ([T2]) for the deterministic IC. Feedback maximizes resource utilization while it cannot 
reduce cost. The "V" curve is obtained when all of the resource levels are fully packed with feedback. This shows the optimality 
of the feedback scheme. 



when < a < 1 (inversely proportional to transmission cost) and must increase when a > 1 
(proportional to the number of resource levels). This is reflected in the V curve. In contrast, in 
the non-feedback case, for some range of a, resource levels are not fully utilized, as shown in 
the a = ^ example of Fig. [6] (a). This is reflected in the W curve. 

Why We Cannot Go Beyond the V Curve: While feedback maximizes resource utilization 
to fill up all of the resource holes, it cannot reduce transmission cost. To see this, consider the 
example in Fig. [6] (6). Observe that even with feedback, a common bit still has to consume two 
levels at the two receivers. For example, the common bit ai needs to occupy the top level at 
receiver 1 in time 1; and the top level at receiver 2 in time 2. In time 1, while ai is received 
cleanly at receiver 1, it interferes with the private bit 62- In order to refine 62, receiver 2 needs 
to get ai cleanly and therefore needs to reserve one resource level for ai. Thus, in order not 
to interfere with the private bit 61, the common bit ai needs to consume a total of the two 
resource levels at the two receivers. As mentioned earlier, assuming that transmission cost is not 
reduced, a total number of transmission bits is reflected in the V curve. As a result, we cannot 
go beyond the "V" curve with feedback, showing the optimality of the achievable scheme. Later 
in Section |Vl we will prove this rigorously. 
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Fig. 8. An Alamouti-based achievable scheme for the Gaussian IC: strong interference regime 



D. An Achievable Scheme for the Gaussian IC 

Let us go back to the Gaussian channel. We will translate the deterministic IC scheme to the 
Gaussian IC. Let us first consider the strong interference regime. 

Strong Interference Regime (INR > SNR): The structure of the transmitted signals in Fig. [5] 
sheds some light on the Gaussian channel. Observe that in the second stage, each transmitter 
sends the other user's information sent in the first stage. This reminds us of Alamouti's scheme 
Q. The beauty of Alamouti's scheme is that received signals can be designed to be orthogonal 
during two time slots, although the signals in the first time slot are sent without any coding. 
This was exploited and pointed out in distributed space-time codes ESll . With Alamouti's 
scheme, transmitters are able to encode their messages so that received signals are orthogonal. 
Orthogonality between the two different signals guarantees complete removal of the interfering 
signal. 

In accordance with the deterministic IC example, the scheme uses two stages (or blocks). 
In the first stage, transmitters 1 and 2 send codewords X^ and X^ with rates Ri and i?2, 
respectively. In the second stage, using feedback, transmitters 1 and 2 decode X^ and X^, 
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respectively. This can be decoded if 



1 



i?i, i?2 < - log (1 + INR) bits/s/Hz. 



(14) 



We are now ready to apply Alamouti's scheme. Transmitters 1 and 2 send X^* and — Xf^*, 
respectively. Receiver 1 can then gather the two received signals: for 1 < i < N, 



(15) 



To extract Xu, it multiplies the row vector orthogonal to the vector associated with X2i and 
therefore we get: 







Qd Qc 




Xii 


+ 


7(1) 
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9d -9c 



(1) 






\9d\ 



(1) 



\g,\')Xu + g:Z',t'-9cZl 



7(2)* 

li ■ 



(16) 



The codeword Xf can be decoded if 



1 



Ri<- log (1 + SNR + INR) bits/s/Hz. 



(17) 



Similar operations are done at receiver 2. Since (fTTl) is implied by ([T4h . we get the desired result: 
the left term in ©. 

Weak Interference Regime (INR < SNR): Unlike the strong interference regime, in the 
weak interference regime, there are two types of information: common and private information. 
A natural idea is to apply Alamouti's scheme only for common information and newly add 
private information. It was shown in ll26ll that this scheme can approximate the symmetric 
capacity to within ^ 1.7 bits/s/Hz. However, the scheme can be improved to reduce the gap 
further. Unlike the deterministic IC, in the Gaussian IC, private signals have some effects, i.e., 
these private signals cannot be completely ignored. Notice that the scheme includes decode-and- 
forward operation at the transmitters after receiving the feedback. And so when each transmitter 
decodes the other user's common message while treating the other user's private signals as noise, 
private signals can incur performance loss. 

This can be avoided by instead performing amplify-and-forward: with feedback, the transmit- 
ters get the interference plus noise and then forward it subject to the power constraints. This 
transmission allows each receiver to refine its corrupted signal sent in the previous time, without 
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causing significant interferenceEl Importantly, notice that this scheme does not require message- 
sphtting. Even without sphtting messages, we can refine the corrupted signals (see Appendix |Al 
to understand this better). Therefore, there is no loss due to private signals. 

Specifically, the scheme uses two stages. In the first stage, each transmitter k sends codeword 
X^ with rate Rk. In the second stage, with feedback transmitter 1 gets the interference plus 
noise: 

S^ = g^X^ + Zj^)'^. (18) 

Now the complex conjugate technique based on Alamouti's scheme is applied to make X^ and 

AT S^* S^* I 

5^2 well separable. Transmitters 1 and 2 send h\^^ and — -7^^=, respectively, where y 1 + INR 
is a normalization factor to meet the power constraint. Under Gaussian input distribution, we can 
compute the rate under MMSE demodulation: i/(Xi^;y/-\y2i )• Straightforward calculations 
give the desired result: the right term in (|7]). See Appendix El for detailed computations. 

Remark 4 (Amplify-and-Forward Reduces the Gap Further): As mentioned earlier, unlike 
the decode-and-forward scheme, the amplify-and-forward scheme does not require message- 
splitting, thereby removing the effect of private signals. This improves the performance to reduce 
the gap further. 

E. An Outer Bound 

Due to the overlap with the outer bound for the capacity region, we defer the proof to 
Theorem [3] in Section IIV-B[ 

F. One-Bit Gap to the Symmetric Capacity 

Using the symmetric rate of (|7]) and the outer bound of ([8]), we get: 

^In Appendix |a1 we provide intuition behind this scheme. 



May 20, 2010 DRAFT 



16 



2(aym - i?sym) < log U + j^^) + log (l + SNR + INR + 2VSNR-INr) 

■(1 + SNR + INR)2-^^ 



1+INR 



1 + 2INR 
1 + SNR + INR 



,1 + SNR + INR + 2VSNR-INR 
1 + INR V 

(1 + 2INR)(1 + INR) 
(1 + SNR + INR)2(1 + INR) - SNR^ 

'1 + SNR + INR + 2VSNR-INR 1 + 2INR \ (19) 

1 + SNR+INR l + INR-(T^smW 



(,) / 2(l + INR-^^^3f^)_i + 

< log ' "^ ^ 



2SNR 



(1+SNR+INR)2 



1 + INR 



SNR 

(1+SNR+INR) 



2 



log 2 . <^ 2 - 



(c) 

< loff4 = 2, 



1 2SNR 

■*- (1+SNR+INR)2 

1 I IMP SNR 

1 -r iiMr\ (i+SNR+INR)2 




where (a) follows from choosing trivial maximum and minimum values of the outer bound ([8]) 
and the lower bound ©, respectively; (6) follows from 1 + SNR + INR + 2VSNR- INR < 
2(1 + SNR + INR); and (c) follows from (1 + SNR + INR)^ > 2SNR and (i^snr+inr)^ < 1- 

Fig. [3] shows a numerical result for the gap between the inner and outer bounds. Notice that 
the gap is upper-bounded by exactly one bit. The worst-case gap occurs when SNR ?^ INR and 
these values go to infinity. Also note that in the strong interference regime, the gap approaches 
with the increase of SNR and INR, while in the weak interference regime, the gap does not 
vanish. For example, when o^ = |, the gap is around 0.5 bits. 

Remark 5 (Why does a l-bit gap occur?): Observe in Figs. [8] and [15] that the transmitted 
signals of the two senders are uncorrelated in our scheme. The scheme completely loses power 
gain (also called beamforming gain). On the other hand, when deriving the outer bound of ([8]), 
we allow for arbitrary correlation between the transmitters. Thus, the 1-bit gap is based on the 
outer bound. In the actual system, correlation is in-between and therefore one can expect that 
an actual gap to the capacity is less than 1 bit. 

Beamforming gain is important only when SNR and INR are quite close, i.e., a ^ 1. This is 
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because when a = 1, the interference channel is equivalent to the multiple access channel where 
Ozarow's scheme [5J and Kramer's scheme [fT5l (that capture beamforming gain) are optimal. 
In fact, the capacity theorem in lfT6ll has shown that Kramer's scheme is optimal for one specific 
case oflNR = SNR — V2SNR, although it is arbitrarily far from optimality for the other cases. 
This observation implies that our proposed scheme can be improved further. 



IV. Capacity Region to Within 2 Bits 
A. An Achievable Rate Region 

We have developed an achievable scheme meant for the symmetric rate and provided a resource 
hole interpretation. For the case of the capacity region, we find that while this interpretation can 
also be useful, the two-staged scheme cannot be applied. A new achievable scheme needs to be 
developed for the region characterization. 

To see this, let us consider a deterministic IC example in Fig. [9] where an infinite number 
of stages need to be employed to achieve a corner point of (2, 1) with feedback. Observe that 



i?2 
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Time 3 Time 2 Time 1 i 

as as ai 
Qq a^ a2 



as 



ai 
&2 h 



as ai 
decode 




Time 2 Time 3 

as ci5 

a4 ® ai ae as 



ai as 

bi ® ai 62 © ^3 bs © as 



I Delay- 



Fig. 9. A deterministic IC example where an infinite number of stages need to be employed to achieve the rate pair of (2, 1) 
with feedback. 

to guarantee i?i = 2, transmitter 1 needs to send 2 bits every time slot. Once transmitter 1 
sends (ai,a2), transmitter 2 cannot use its top level since the transmission causes interference 
to receiver 1. It can use only the bottom level to send information. This transmission however 
suffers from interference: receiver 2 gets the interfered signal 6i © ai. We will show that this 
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corrupted bit can be refined with feedback. In time 2, transmitter 2 can decode ai with feedback. 
In an effort to achieve the rate pair of (2, 1), transmitter 1 sends (as, a4) and transmitter 2 sends 
62 on the bottom level. Now apply the same idea used in the symmetric case: transmitter 2 sends 
the other user's information ai on the top level. This transmission allows receiver 2 to refine 
the corrupted signal 61 without causing interference to receiver 1, since receiver 1 already had 
ai as side information. Notice that during the two time slots, receiver 1 can decode 4 bits (2 
bits/time), while receiver 2 can decode 1 bits (0.5 bits/time). The point (2, 1) is not achieved yet 
due to unavoidable loss occurred in time 1. This loss, however, can be amortized by iterating the 
same operation. As this example shows, the previous two-staged scheme needs to be modified 
so as to incorporate an infinite number of stages. 

Let us apply this idea to the Gaussian channel. The use of an infinite number of stages 
motivates the need for employing block Markov encoding [191, HIOH . Similar to the symmetric case, 
we can now think of two possible schemes: (1) decode-and-forward (with message-splitting); 
and (2) amplify-and-forward (without message-splitting). As pointed out in Remark IH in the 
Gaussian channel, private signals cannot be completely ignored, thereby incurring performance 
loss, thus the amplify-and-forward scheme without message-splitting has better performance. 
However, it requires heavy computations to compute the rate region, so we focus on the decode- 
and-forward scheme, although it induces a larger gap. As for a decoding operation, we employ 
backward decoding [El, [|27l. 

Here is the outline of our scheme. We employ block Markov encoding with a total size B of 
blocks. In block 1, each transmitter splits its own message into common and private parts and 
then sends a codeword superimposing the common and private messages. For power splitting, we 
adapt the idea of the simplified Han-Kobayashi scheme ^ where private power is set such that a 
private signal is seen below the noise level at the other receiver. In block 2, with feedback, each 
transmitter decodes the other user's common message (sent in block 1) while treating the other 
user's private signal as noise. Two common messages are then available at the transmitter: (1) its 
own message; and (2) the other user's message decoded with the help of feedback. Conditioned 
on these two common messages, each transmitter generates new common and private messages. It 
then sends the corresponding codeword. Each transmitter repeats this procedure until block B — 1. 
In the last block B, to facilitate backward decoding, each transmitter sends the predetermined 
common message and a new private message. Each receiver waits until total B blocks have been 
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received and then performs backward decoding. We will show that this scheme enables us to 
obtain an achievable rate region that approximates the capacity region. 

Theorem 2: The feedback capacity region includes the set TZ of (i?i, R2) such that 

Ri < log (1 + SNRi + INR21 + 2pVSNRi ■ INR21) - 1 (20) 

i?i < log (1 + (1 - p)INRi2) + log (2 + ^^] - 2 (21) 

R2 < log (1 + SNR2 + INR12 + 2pVSNR2 • INR12) - 1 (22) 

i?2 < log (1 + (1 - p)INR2i) + log (2 + ^"j - 2 (23) 

Ri+R2< log ^2 + ^^j + log (1 + SNR2 + INR12 + 2pVSNR2 • INR12) - 2 (24) 

Ri + R2< log (2 + ^^) + log (1 + SNRi + INR21 + 2pVSNRi ■ INR21) - 2 (25) 

for < p < 1. 

Proof: Our achievable scheme is generic, not limited to the Gaussian IC. We therefore 
characterize an achievable rate region for discrete memoryless ICs and then choose an appropriate 
joint distribution to obtain the desired result. In fact, this generic scheme can also be applied to 
El Gamal-Costa deterministic IC (to be described in Section JV)). 

Lemma 1: The feedback capacity region of the two-user discrete memoryless IC includes the 
set of (i?i, i?2) such that 

i?i</(f/,C/2,Xi;Fi) (26) 

Ri < I{Ui-Y2\U,X2) + I{X^-,Y^\Ui,U2^U) (27) 

i?2</(f/,f/i,X2;F2) (28) 

R2 < I{U2;Y,\U,X,) + I{X2,Y2\Ui,U2^U) (29) 

i?i + i?2 < /(Xi; n|t/i, f/2, U) + /(t/, [/i, X2; Y2) (30) 

i?i + i?2 < /(X2; yalC/i, t/2, U) + /([/, f/2, Xi; Fi), (31) 

over all joint distributions ]9(t^)]9(2ii|2i)]9(2i2|'^)p(^i|'^i, t^)p(x2|t^2, '^)- 

Proof: See Appendix |Bl ■ 
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Now we will choose the following Gaussian input distribution to complete the proof: V/c = 1, 2, 

U - CA/"(0, p); [/, - CA/"(0, A,,); X^, - CA/'(0, A^,), (32) 

where Xj. = [/ + t/^t + -^fc^; Kk and A^/e indicate the powers allocated to the common and 
private message of transmitter k, respectively; and ([/, Uk, Xj^pYs are independent. By symmetry, 
it suffices to prove (EQI), (EB and (1241) . 

To prove ([20l), consider /([/, t/s, Xi; Yi) = h{Yi) - h{Yi\U, U2, Xi), Note 

|i^n|Xi,t/2,t/| = l + A^2lNR2i. (33) 

As mentioned earlier, for power splitting, we adapt the idea of the simplified Han-Kobayashi 
scheme ^. We set private power such that the private signal appears below the noise level at 
the other receiver. This idea mimics that of the deterministic IC example where the private bit 
is below the noise level so that it is invisible. The remaining power is assigned to the common 
message. Specifically, we set: 

A^2 = min f — — — ,1 j , Ac2 = 1 - A^2, (34) 

This choice gives 

/([/, [/2, Xi; Yi) = log (1 + SNRi + INR21 + 2pVSNRi • INR21) - 1, (35) 

which proves ( |20l) . With the same power setting, we can compute: 

/([/i;y2|t/,X2) = log(l + (l-p)INRi2)-l, (36) 

/(Xi;yi|[/,[/i,[/2) = log (2 + ^) - 1. (37) 

This proves (EI]). Lastly, by ([35]) and ([37]), we prove (|24|). ■ 

Remark 6 (Three Types of Inequalities): In the non-feedback case, it is shown in ^ that an 
approximate capacity region is characterized by five types of inequalities including the bounds for 
2R1 + R2 and R1 + 2R2. In contrast, in the feedback case, our achievable rate region is described 
by only three types of inequalities E| In Section IVI-B[ we will provide qualitative insights as to 
why the 2Ri + R2 bound is missing with feedback. 

^It is still unknown whether or not the exact feedback capacity region includes only three types of inequalities. 
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Remark 7 (Connection to Related Work l{T7\]): Our achievable scheme is essentially the same 
as the scheme introduced by Tuninetti [J27j| in a sense that the three techniques (message-splitting, 
block Markov encoding and backward decoding) are jointly employ edQ However, the scheme 
in [|27l uses five auxiliary random variables requiring further optimization. On the other hand, 
we obtain an explicit rate region by reducing those five auxiliary random variables into three 
and then choosing a joint input distribution appropriately. 

B. An Outer Bound Region 

Theorem 3: The feedback capacity region is included by the set C of (i?i, R2) such that 

Ri < log (1 + SNRi + INR21 + 2pVSNRi • INR21) (38) 

H. < log (1 . (1 - /),NR.) + log (1 + rf(T^^) (39) 

i?2 < log (1 + SNR2 + INR12 + 2pVSNR2 • INR12) (40) 

H. < log (1 . (1 - /)INR.) + log (1 + rrO^I^) (41) 

Ri + R2< log (1 + ^|\~^^2)')'nr^J + log (1 + SNR2 + INR12 + 2pVSNR2-INRi2) 

(42) 

Ri + R2< log (1 + ^ |\~ 2)*)*^ R, J + 1°S (1 + SNRi + INR21 + 2pVSNRi • INR21) 

(43) 
for < p < 1. 

Proof: By symmetry, it suffices to prove the bounds of (l38l) . (l39b and (|42l) . The bounds of 
(|38l) and (l39b are nothing but cutset bounds. Hence, proving the non-cutset bound of (|42|) is the 
main focus of this proof. Also recall that this non-cutset bound is used to obtain the outer bound 
of dSl) for the symmetric capacity in Theorem [U We go through the proof of (l38b and (l39l) . We 
then focus on the proof of (|42|) . where we will also provide insights as to the proof idea. 
Proof of (l38b ; Starting with Fano's inequality, we get: 

N{Ri - er,) < I{Wi-Y,'') < J2[h{Yu) - h{Zu)], 

^The author in |[27l considers a different context: the conferencing encoder problem. However, Prabhakaran and Viswanath |[T9l 
have made an interesting connection between the feedback problem and the conferencing encoder problem. See 1 19| for details. 
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where (a) follows from the fact that conditioning reduces entropy. Assume that Xi and X2 have 
covariance p, i.e., ^'[XiXg] = p. Then, we get: 

HYi) < log27re (l + SNRi + INR21 + 2|p|VSNRi • INR21) . (44) 

If (i?i, R2) is achievable, then cat ^ as A/^ ^ 00. Therefore, we get the desired bound: 

Ri < h{Yi) - h{Zi) < log (1 + SNRi + INR21 + 2|p|VSNRi-INR2i) . (45) 

Proof of (139b ; Starting with Fano's inequality, we get: 

N{Ri - e^) < I{Wi;Y,'',Yf,W2) 

^^ Y,[hiYu,Y2i\W2,Yt\Yt') - HZu) - /i(^2i)] 
^^ J2[HYu,Y2.\W2,Yt\Yt\Xi) - h{Zu) - h{Z2^)] 

^^ Y}Ky2i\w2X~\Yt\xi)-h{Z2i)] + Y}^^^^^\^^^"^t\yt\xiY2uS^^^ 

< Y, [Hy2i\X2i) - h{Z2i) + h{Yu\X2i, Sii) - h{Zu)] 

where (a) follows from the fact that Wi is independent from W2 and h{Y/^ ,Y^\Wi,W2) = 
h{Yf, S^\Wi, W2) = 'ElHZii) + h{Z2i)] (see Claim©; (b) follows from the fact that X^ is a 
function of (^2,^2"^)^ ('^) follows from the fact that SI is a function of {Y2,X2); (d) follows 
from the fact that conditioning reduces entropy. Hence, we get the desired result: 

Ri < h{Y2\X2) - h{Z2) + h{Y^\X2, Si) - h{Zi) 

where (a) follows from the fact that 

h{Y2\X2) < log27re (l + (1 - |pniNRi2) , (46) 

The inequality of (l47l) is obtained as follows. Given (X2, Si), the variance of Yi is upper-bounded 
by 

Var[yi|X2,^i] < Ky, - Ky,^x,,s,)Kll^s,)Kkix,,s,), 
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where 



Ky, = E [iFil^] = 1 + SNRi + INR21 + 991^921 + P*gii9*2i, 
Ky^{X2,Si) = E [Yi[X*, S^]] = [pgii + 921,9*12911 + P* 9219*12] > 



K, 



{X2,Sl) 



= E 



1 P*9*12 

pgi2 I + INR12 



(48) 



I V 1^ V C* 

x;s, \S,\' 

By further calculation, we can get (l47l) . 

Proof of ([42b : The proof idea is based on the genie-aided argument [fT3]l . However, finding 
an appropriate genie is not simple since there are many possible combinations of the random 
variables. The deterministic IC example in Fig. [6] (6) gives insights into this. Note that providing 
ai and (61, 62,^3) to receiver 1 does not increase the rate i?i, i.e., these are useless gifts. This 
may motivate us to choose a genie as (^'12X1, W2). However, in the Gaussian channel, providing 
^'12X1 is equivalent to providing Xi. This is of course too much information, inducing a loose 
upper bound. Inspired by the technique in [[8]], we instead consider a noisy version of ^'12-^1: 



5^1 — guXi + Z2. 



(49) 



Intuition behind this is that we cut off ^'12-^1 at the noise level. Indeed this matches intuition 
in the deterministic IC. This genie together with W2 turns out to lead to the desired tight upper 
bound. 

On top of the genie-aided argument, we need more techniques. In the feedback problem, the 
functional relationship between the random variables is more complicated and thus needs be 
well explored. We identify several functional relationships through Claims [H [2l and [3l which 
turn out to play a significant role in proving the non-cutset bound of (l42h . 
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Starting with Fano's inequality, we get: 

7V(i?i + i?2 - eiv) < I{Wu Fi^) + I{W2; Y,"") 

^=^ /i(n^, 5f IW-2) - hiY,"", S^\Wi, W2) + I{W2; Yf) 
'd hiY,"", S^\W2) - Y. [h{Zu) + HZ^i)] + IiW2; Yf) 
^^ /^(n^l^f , W2) - J2 h{Zu) + KYi") - J2 KZ2{) 
^^ /^(n^l^f , W,, X^) - J2 h{Zu) + HYf) - J2 h{Z2^) 

(/) ^ 

< J2 [HYu\Su, X2i) - h{Zu) + h{Y2i) - h{Z2i)] 

i=l 

where (a) follows from the fact that adding information increases mutual information (providing a 
genie); (b) follows from the independence of Wi and W2', (c) follows from h{Yf, S^\Wi, W2) = 
^ [h[Zu) + KZ2i)] (see Claim©; [d) follows from h{S^\W2) = h{Yf\W2) (see ClaimO; (e) 
follows from the fact that X^ is a function of {W2, S^'^) (see Claim [S]); (/) follows from the 
fact that conditioning reduces entropy. 
Hence, we get 

Ri + R2< h{Yi\Si, X2) - h{Zr) + h{Y2) - KZ2). 

Note that 

KY2) < log27re (1 + SNR2 + INR12 + 2|p| VSNR2 • INR12) . (50) 

From (l47l) and ([50b . we get the desired upper bound. 
Claim 1: h{Yf , S^\Wi, W2) = E iKZu) + h{Z2^)] . 
Proof: 

hiY,'',S^\WuW2) = J2h(^ii^Su\WuW2,Yr\Si-') 

^=^ Y, KZli, Z2i\Wi, W2, YI-\ Sl-\ Xu, X2^) 

i J2 HZii) + KZ2i)] , 

where (a) follows from the fact that Xu is a function of {Wi,Yl~^) and ^2^ is a function 
of (VI/2, S[~^) (by Claim [S]); (6) follows from the fact that Yu = guXu + ^21^2^ + Zu and 
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Sii = gi2Xii+Z2i', (c) follows from the memoryless property of the channel and the independence 
assumption of Zn and Z2i. ■ 

Claim 2: h{S^\W2) = h{Yf\W2). 
Proof: 

^^2^11^2) = j]M^2iii^r\ W2) 

i 5]/i(^H|Vr2,5r') = h{S^\W2), 
where (a) follows from the fact that Y2i is a function of {X2i,Sii) and ^2^ is a function of 
(VI/2, ^2"^); (b) follows from the fact that X^ is a function of {W2, Y^~^) and S'i"^ is a function 
of (y2'"\^r^); (c) follows from the fact that Y^'^ is a function of {X'^^Sl'^) and X^ is a 
function of (1^2, ^i"^) (by Claim [3]). ■ 

Cte/m 3; For all i > 1, Xj is a function of {Wi, Si~^) and X^ is a function of (VI/2, 5^^"^). 
Proof: By symmetry, it is enough to prove only one. Notice that X^ is a function of 
{W2, Y^-^) and Y^'^ is a function of {X'f\ S{-^). Hence, X^ is a function of (1^2, ^^^ ^i"^)- 
Iterating the same argument, we conclude that X2 is a function of {W2,X2i, S{~^). Since X21 
depends only on W2, we complete the proof. ■ 



C. 2-Bit Gap to the Capacity Region 

Theorem 4: The gap between the inner and upper bound regions (given in Theorems [21 and [3]) 
is at most 2 bits/s/Hz/user: 

7^ccccc7^e ([o, 2] x [0, 2]) . (5i) 

Proof: The proof is immediate by Theorem [21 and [3l We define 5i to be the difference 
between min { (J38l) . ([39l) } and min { (J20l) . fJ2T1)}. Similarly, we define ^2 and 5i2. Straightforward 
computation gives 

Similarly, we get S2 < 2 and 5i2 < 2. This completes the proof. ■ 
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Remark 8 (Why does a 2-bit gap occur?): The achievable scheme meant for the capacity 
region involves message-splitting. As mentioned in Remark IH message-splitting incurs some 
loss in the process of decoding the common message while treating private signals as noise. 
Accounting for the effect of private signals, the effective noise power becomes double, thus 
incurring a 1-bit gap. The other 1-bit gap comes from a relay structure of the feedback IC. To 
see this, consider an extreme case where user 2's rate is completely ignored. In this case, we 
can view the [transmitter 2, receiver 2] communication pair as a single relay which only helps 
the [transmitter!, receiver 1] communication pair. It has been shown in H that for this single 
relay Gaussian channel, the worst-case gap between the best known inner bound [l9l| and the 
outer bound is 1 bit/s/Hz. This incurs the other 1-bit gap. This 2-bit gap is based on the outer 
bound region in Theorem [3l which allows for arbitrary correlation between the transmitters. So, 
one can expect that an actual gap to the capacity region is less than 2 bits. 

Remark 9 (Reducing the gap): As discussed, the amplify-and-forward scheme has the po- 
tential to reduce the gap. However, due to the inherent relay structure, reducing the gap into a 
less-than-one bit is challenging. As long as no significant progress is made on the single relay 
Gaussian channel, one cannot easily reduce the gap further. 

Remark 10 (Comparison with the two-staged scheme): Specializing to the symmetric rate, it 
can be shown that the infinite-stagtd scheme in Theorem [2] can achieve the symmetric capacity 
to within 1 bit. Coincidentally, this gap matches the gap result of the m^6^-staged scheme in 
Theorem [B However, the 1-bit gap comes from different reasons. In the infinite- staged scheme, 
the 1-bit gap comes from message-splitting. In contrast, in the two-staged scheme, the gap is due 
to lack of beamforming gain. One needs to come up with a new technique that well combines 
these two schemes to reduce the gap into a less-than-one bit. 

V. The Feedback Capacity Region of El Gamal-Costa Model 

We have so far made use of the linear deterministic IC to provide insights into approximating 
the feedback capacity region of the Gaussian IC. The deterministic IC is a special case of El 
Gamal-Costa deterministic IC [fT3]| . In this section, we establish the exact feedback capacity 
region for this general class of deterministic ICs. 

Fig. [To] (a) illustrates El Gamal-Costa deterministic IC with feedback. The key condition of 
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Fig. 10. El Gamal-Costa deterministic IC with feedback 

this model is given by 

//(y2|n,Xi) = o, 

H{V^\Y2,X2) = Q, 
where Vk is a part of X^ (k = 1,2), visible to the other receiver. This implies that in any 
working system where Xi and X2 are decodable at receivers 1 and 2 respectively, Vi and V2 
are completely determined at receivers 2 and 1, respectively, i.e., these are common signals. 

Theorem 5: The feedback capacity region of El Gamal-Costa deterministic IC is the set of 
{Ri, R2) such that 

R, < min {H{Y,), HiY^lX^, U) + H{Y^\V^, V^^ U)} 
R2 < min {H{Y2), H{Y,\X,, U) + //(Fsl^i, ^2, U)} 
Ri + R2< min {H{Y,\Vi, V2, U) + //(Fs), H{Y2\V2^ ^i, U) + H{Y,)} 

for some joint distribution p{u, xi,X2) = p{u)p{xi\u)p{x2\u). Here [/ is a discrete random 
variable which takes on values in the set U where \U\ < min(|Vi||V2|, |3^i|, |3^2|) + 3. 

Proof: Achievability proof is straightforward by Lemma [B Let Uk = Vk^k. Fix a joint dis- 
tribution p(t^)p(2ii|2i)p(t^2|'^)p(^i I '^1, u)p{x2\u2, u). We now write a joint distribution p(2i, xi, X2, t^i, U2) 
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in two different ways: 

= p{u)p{xi\u)p{x2\u)S{ui - gi{xi))S{u2 - ^2(^2) (53) 

= p{u)p{ui\u)p{u2\u)p{xi\ui, U)p{x2\u2, u) 

where 5() indicates the Kronecker delta function. This gives 

p{xi\ui^ u)p{ui\u) 



p{xi\u) :■ 
p{x2\u) : = 



6{ui - gi{xi)) 

P{X2\U2,U)P{U2\U) 



(54) 



5(^2-^2(^2)) 
Now we can generate a joint distribution p{u)p{xi\u)p{x2\u). Hence, we complete the achiev- 

ability proof. See Appendix [0 for converse proof. ■ 

As a by-product, we obtain the feedback capacity region of the linear deterministic IC. 
Corollary 1: The feedback capacity region of the linear deterministic IC is the set of (i?i, R2) 

such that 

Ri < min{max(nii, 77,12), max(nii, 77.21)} 

i?2 < min{max(n22,^2i),niax(n22,^i2)} 

Ri + R2 < min {max(ri22, ^12) + (^11 - ^12)^, max(nii, 77.21) + (^22 - ^21)^} • 

Proof: The proof is straightforward by Theorem El The capacity region is achieved when 
U is constant; and Xi and X2 are independent and uniformly distributed. ■ 

VI. Role of Feedback 

Recall in Fig. [I] that feedback gain is bounded for < a < | in terms of the symmetric rate. 
So a natural question arises: is feedback gain marginal also from a perspective of the capacity 
region? With the help of Corollary [H we show that feedback can provide multiplicative gain 
even in this regime. We also answer another interesting question posed in Section [IVl why is 
the 2i?i + i?2 bound missing with feedback? 
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Fig. 11. Feedback capacity region of the linear deterministic IC 

A. Feedback Gain from a Capacity Region Perspective 

Fig. [n] shows the feedback capacity region of the hnear deterministic IC under the symmetric 
channel setting: n = nn = 7122 and m = ni2 = ^21. Interestingly, while for | < a < 2, the 
symmetric capacity does not improve with feedback, the feedback capacity region is enlarged 
even for this regime. 

B. Why is 2Ri + R2 Bound Missing with Feedback? 

Consider an example where 2Ri + R2 bound is active in the non-feedback case. Fig. [I2l (a) 
shows an example where a corner point of (3, 0) can be achieved. Observe that at the two 
receivers, the five signal levels are consumed out of the six signal levels. There is one resource 
hole. This resource hole is closely related to the 2R1 + R2 bound, which will be shown in Fig. [121 
(6). 

Suppose the 2Ri + R2 bound is active. This implies that if Ri is reduced by 1 bit, then 
i?2 should be increased by 2 bits. To decrease Ri by 1 bit, suppose that transmitter 1 sends 
no information on the second signal level. We then see the two empty signal levels at the two 
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(a) A resource hole vs. 2Ri -\- R2 bound 



Must b e a resource hole. 

{b) Whenever 2Ri + R2 is active, there is a resource hole. 
R2 
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(c) Feedback fills up all resource holes to maximize resource utilization 

Fig. 12. Relationship between a resource hole and 2Ri + R2 bound. The 2Ri + R2 bound is missing with feedback. 

receivers (marked as the gray balls): one at the second level at receiver 1; the other at the bottom 
level at receiver 2. Now transmitter 2 can send 1 bit on the bottom level to increase R2 by 1 
bit (marked as the thick red line). Also it allows transmitter 2 to send one more bit on the top 
level. This implies that the top level at receiver 2 must be a resource hole in the previous case. 
This observation combined with the following observation can give an answer to the question. 
Fig. [12] (c) shows the feedback role that it fills up all the resource holes to maximize resource 
utilization. We employ the same feedback strategy used in Fig. [9] to obtain the result in Fig. [12] 
(c). Notice that with feedback, all of the resource holes are filled up except a hole in the first 
stage, which can be amortized by employing an infinite number of stages. Therefore, we can 
now see why the 2Ri + R2 bound is missing with feedback. 
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VII. Discussion 
A. Comparison to Related Work / [75l/ . / [76l/ . / [77l/ 

For the symmetric Gaussian IC, Kramer ifTSll , lfT6ll developed a feedback strategy based on 
Schalkwijk-Kailath scheme ll28]l and Ozarow's scheme [5J. Due to lack of closed-form rate- 
formula for the scheme, we cannot see how Kramer's scheme is close to our symmetric rate in 
Theorem [H To see this, we compute the generalized degrees-of-freedom of Kramer's scheme. 

Lemma 2: The generalized degrees-of-freedom of Kramer's scheme is given by 



1 — a, < a < |; 



d{a) = { ^ 



4 ' 



Proof: See Appendix |Dl 



I < a < 1; 
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Fig. 13. Generalized degrees-of-freedom comparison 



Fig. 14. The symmetric rate comparison 



Note in Fig. [13] that Kramer's scheme can be arbitrarily far from optimality, i.e., it has an 
unbounded gap to the symmetric capacity for all values of a except a = 1. We also plot the 
symmetric rate for finite channel parameters as shown in Fig. [TH Notice that Kramer's scheme 
is very close to the outer bounds only when INR is similar to SNR. In fact, the capacity theorem 



in ^^ says that they match each other at INR = SNR- V2SNR. However, if INR is quite 
different from SNR, it becomes far away from the outer bounds. Also note that our new bound 
is much tighter than Gastpar-Kramer' s outer bounds in llTSll . ifTTll . 
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B. Closing the Gap 

Less than 1-bit gap to the symmetric capacity: Fig. [14] implies that our achievable scheme 
can be improved especially when a ^ 1 where beamforming gain plays a significant role. 
As mentioned earlier, our two-staged scheme completely loses beamforming gain. In contrast, 
Kramer's scheme captures the beamforming gain. As discussed in Remark [TOl one may develop 
a unified scheme that beats both the schemes for all channel parameters to reduce the worst-case 

gap. 

Less than 2-bit gap to the capacity region: As mentioned in Remark [8l a 2-bit gap to the 
feedback capacity region can be improved up to a 1-bit gap. The idea is to remove message 
splitting. Recall that the Alamouti-based amplify-and-forward scheme in Theorem [H improves 
the performance by removing message splitting. Translating the same idea to the characterization 
of the capacity region is needed for the improvement. A noisy binary expansion model in Fig. [16] 
may give insights into this. 

C Extension to Gaussian MIMO ICs with Feedback 

The feedback capacity result for El Gamal-Costa model can be extended to Teletar-Tse IC [[T4]| 
where in Fig. [TOl //e's are deterministic functions satisfying El Gamal-Costa condition ([52h while 
Qk^ follow arbitrary probability distributions. Once extended, one can infer an approximate 
feedback capacity region of the two-user Gaussian MIMO IC, as [[T4]| did in the non-feedback 
case. 

VIII. Conclusion 

We have established the feedback capacity region to within 2 bits/s/Hz/user and the symmetric 
capacity to within 1 bit/s/Hz/user universally for the two-user Gaussian IC with feedback. 
Alamouti's scheme inspires our two-staged achievable scheme meant for the symmetric rate. For 
an achievable rate region, we have employed block Markov encoding to incorporate an infinite 
number of stages. A new outer bound was derived to provide an approximate characterization of 
the capacity region. As a side-generalization, we have characterized the exact feedback capacity 
region of El Gamal-Costa deterministic IC. 

An interesting consequence of our result is that feedback could provide multiplicative gain in 
many-to-many channels unlike point-to-point, many-to-one, or one-to-many channels. Developing 
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a resource hole interpretation, we have provided quahtative insights as to how feedback can 
provide significant gain even in the weak interference regime where there is no better alternative 
path. We have shown that feedback maximizes resource utilization to provide the gain. 

Appendix A 
Achievable Scheme for the Symmetric Rate of ([7]) 



get S2 



Block 2 
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Vl + INR 
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Fig. 15. An achievable scheme in the symmetric Gaussian IC: Alamouti-based amplify- and-forward scheme 



The scheme uses two stages (blocks). In the first stage, each transmitter k sends codeword 
X^ with rate Rk. In the second stage, with feedback transmitter 1 gets the interference plus 



noise: S2 



^W,N 



QcX^ +Z{ ^' . Now the complex conjugate technique based on Alamouti's scheme 



is applied to make Xf and 5^2 well separable. Transmitters 1 and 2 send 



s^^ 



Vl+INR 



and 



s^ 



Vl+INR' 



respectively, where Vl + INR is a normalization factor to meet the power constraint. 
Receiver 1 can then gather the two received signals: for 1 < i < A^, 
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Under Gaussian input distribution, we can compute the rate under MMSE demodulation: 



^/(Xh; Y,) = \h{Y,) - \h{Y,\Xu) = \\o^ 



Yi|Xii I 



(56) 



(57) 
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Straightforward calculations give 

I ^ I \ 



1 + SNR + INR 

9d I 



9d 



\K- 



Yi|Xij I 



Vl+INR 

1 + INR 



Vl+INR 

SNR + INR 



= (1 + SNR + INR)^ 



SNR 
1 + INR 



Wl + INR 
g^VlTmR SNR + f^ 

Therefore, we get the desired result: the right term in ©. 



1 + 2INR. 
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Zi's i.i.d. ~ Ber(l) 

(a) A noisy binary-expansion model 
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(6) Interpretation of S chalk wijk-Kailath scheme 



Fig. 16. A noisy binary expansion model. Noise is assumed to be a Ber(|) random variable i.i.d. across time slots (memoryless) 
and levels. This induces the same capacity as that of the deterministic channel, so it matches the Gaussian channel capacity in 
the high SNR regime. 

Intuition Behind the Proposed Scheme: To provide intuition behind our proposed scheme, 
we introduce a new model what we call a noisy binary expansion model, illustrated in Fig. [16] 
(a). In the non-feedback Gaussian channel, due to the absence of noise information at transmitter, 
transmitter has no chance to refine the corrupted received signal. On the other hand, if feedback 
is allowed, noise can be learned. Sending noise information (innovation) can refine the corrupted 
signal: Schalkwijk-Kailath scheme Il28ll . However, a linear deterministic model cannot capture 
interplay between noise and signal. To capture this issue, we slightly modify the deterministic 
model so as to reflect the effect of noise. In this model, we assume that noise is a Ber(|) random 
variable i.i.d. across time slots (memoryless) and levels. This induces the same capacity as that of 
the deterministic channel, so it matches the Gaussian channel capacity in the high SNR regime. 
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Time 2 




a2 ® 4^^ ® ^1 



h®a2® z9-^ 



.(2) 



g^* a7e.?^ 



Fig. 17. Intuition behind the Alamouti-based ampUfy-and-forward scheme. 

As a stepping stone towards the interpretation of the proposed scheme, let us first understand 
Schalkwijk-Kailath scheme II28I1 using this model. Fig. [16] (6) illustrates an example where 
2 bits/time can be sent with feedback. In time 1, transmitter sends independent bit streams 
(ai, a2, a3, a4, • • • ). Receiver then gets {ai, a2, a^ ® z[ \ a^ ® Z2 r • • ) where z--^^ indicates an 
i.i.d. Ber (|) random variable of noise level i at time j. With feedback, transmitter can get noise 
information (0, 0, z^ \ ^2 , • • • ) by subtracting the transmitted signals (sent previously) from 
the received feedback. This process corresponds to an MMSE operation in Schalkwijk-Kailath 
scheme: computing innovation. Transmitter scales the noise information to shift it by 2 levels 
and then sends the shifted version. The shifting operation corresponds to a scaling operation in 
Schalkwijk-Kailath scheme. Receiver can now recover (as.a^) corrupted by (^ , ^2 ) ^^ the 
previous slot. We repeat this procedure. 

The viewpoint based on a binary expansion model can provide intuition behind our proposed 
scheme. See Fig. [TTl In the first stage, each transmitter sends three independent bits: two bits 
above the noise level; one bit below the noise level. Transmitters 1 and 2 send {ai,a2,as) 
and (61, 62,^3), respectively. Receiver 1 then gets: (1) the clean signal ai; (2) the interfered 
signal a2 © 61; and (3) the interfered-and-noised signal as 62 ® ^1 • Similarly for receiver 
2. In the second stage, with feedback, each transmitter can get interference plus noise by 
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subtracting the transmitted signals from the feedback. Transmitters 1 and 2 get (0, bi,b2® z[ ^) 
and (0, ai, a2 © ^2 )' respectively. Next, each transmitter scales the subtracted signal subject to 
the power constraint and then forwards the scaled signal. Transmitters 1 and 2 send (6i, b2®z[ ^) 
and (ai, a2 ©^2 )' respectively. Each receiver can then gather the two received signals to decode 
3 bits. From this figure, one can see that it is not needed to send additional information on top 
of innovation in the second stage. Therefore, this scheme matches Alamouti-based amplify-and- 
forward scheme in the Gaussian channel. 

Appendix B 
Proof of Lemma \T\ 

Codebook Generation: Fix a joint distribution p{u)p{ui\u)p{u2\u)p{xi\ui, u)p{x2\u2, u). First 
generate 2^(^i-+^2c) independent codewords u^{ij), i G {1, • • • , 2^^i^}, j G {1, • • • , 2^^^-}, 
according to Yli=iP{ui). For each codeword u^{i,j), encoder 1 generates 2^^^^ indepen- 
dent codewords u^{{i,j),k), k G {1, • ' ' ,2^^^^}, according to Ylf=iP{uii\ui). Subsequently, 
for each pair of codewords {u^{i,j),u^{{i,j),k)), generate 2^^^^ independent codewords 
xf((i,j),/c,/), / G {!,••• ,2^^!^}, according to ]\^^^p{xii\uii,Ui). 

Similarly, for each codeword u^{i,j), encoder 2 generates 2^^2c independent codewords 
^2{{h3)^r), r e {!,••• ,2^^^^}, according to ]\^=iP{u2i\ui), For {u^{i,j),u^{{i,j),r)), gen- 
erate 2^^2p independent codewords x^((i, j), r, 5), 5 G {1, • • • , 2^^'^tp}, according to Y\i=i P{^2i\u2i, ui). 

Notation: Notations are independently used only for this section. The index k indicates the 
common message of user 1 instead of user index. The index i is used for both purposes: (1) 
indicating the previous common message of user 1 ; (2) indicating time index. It could be easily 
differentiated from contexts. 

Encoding and Decoding: We employ block Markov encoding with a total size B of blocks. 
Focus on the 6th block transmission. With feedback y-^'^ ~ \ transmitter 1 tries to decode the 
message W2~ = k (sent from transmitter 2 in the {h — l)th block). In other words, we find the 
unique k such that 



Ic 



Un 
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where Ae indicates the set of jointly typical sequences. Note that transmitter 1 already knows 
its own messages {w[~ \w[~ \w[~ ^). We assume that W2~ is correctly decoded from the 
previous block (6—1). The decoding error occurs if one of two events happens: (1) there is no 
typical sequence; (2) there is another W2~ ^^^h that it is a typical sequence. By AEP, the first 
error probability becomes negligible as A^ goes to infinity. By [|29l1 . the second error probability 
becomes arbitrarily small (as A^ goes to infinity) if 

R2c<I{U2;Y,\X,,U). (60) 

Based on {w[~ ,^2c" )' transmitter 1 generates a new common message w{J and a private 
message wlj. It then sends x^ i{w[~ \w2~ ),w[^ ,w[p]. Similarly transmitter 2 decodes 
w{~^\ generates {wf^,W2p) and then sends x^ U'^u\^2c)^^2^ 

Each receiver waits until total B blocks have been received and then does backward decoding. 
Notice that a block index b starts from the last B and ends to 1. For block 6, receiver 1 finds 
the unique triple (i, j, k) such that 

(zi^ (l,j) ,< ((z,]),i&S?) ,xf {ilj),wf^,k) , 

where we assumed that a pair of messages (^L ^^L) was successively decoded from block 
(6+1). Similarly receiver 2 decodes {w[~ , t&2c" ^ '^2p )• 

Error Probability: By symmetry, we consider the probability of error only for block b and 
for a pair of transmitter 1 and receiver 1. We assume that {wl~ ,^2c" \^ip) = (1^ 1^ 1) was 
sent through block {b — 1) and block 6; and there was no backward decoding error from block 
B io {b + 1), i.e., (t&L ^^L) ^^^ successfully decoded. 

Define an event: 

Eijk = |(^^^(i,i),^f((i,i)>fj),xf((i,j)>i?,/c), 

By AEP, the first type of error becomes negligible. Hence, we focus only on the second type of 
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error. Using the union bound, we get 



Pr I U E,,j, 

,(i,j,/c)/(l,l,l) 



i/l,j/l,/c/l i/l,j/l,fc=l i/l,j=l,A:/l 



z/l,j=l,A^=l i=l,j/l,fc/l i=l,j/l,fc=l i=lj=l,/c/l 

_^ 2^(^ic-^(^'^i'^2;n)+4e) _^ 2^(^2c+i^ip-/(t/,Xi,[/2;yi)+4e) _^ 2^(^2c-/(f/,^i,t/2;n)+4e) 
_^ 2^(^ip-^(^i;^i|^'f^i'^2)+4e) 

(61) 
From (l60l) and (|6T1) , we can say that the error probabihty can be made arbitrarily small if 

R2c <I{U2;Y,\X,,U) 

Rip </(Xi;yi|[/i,[/2,[/) (62) 

Ric <I{U,;Y2\X2,U) 

R2p <I{X2',Y2\U^,U2^U) (63) 

R2c + R2p + Ric <HU,X2,Ui;Y2). 
Fourier-Motzkin Elimination: Applying Fourier-Motzkin elimination, we easily obtain the 
desired inequalities. There are several steps to remove Rip, i?2p, Ric^ and i?2c. successively. First 
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substitute Rip = Ri — Ric and i?2p = R2 — R2C to get: 

R2c<I{U2;Yi\Xi,U) 
Ri-Ri,<I{Xi;Yi\Ui,U2,U) 
Ri + R2c<I{U,Xi,U2;Yi) 

Ric<I{Ui;Y2\X,,U) 

R2-R2c<I{X2;Y2\Ui,U2,U) 

R2 + Ric<I{U,X,,Ui-Y2) 
-Ric < 
-Ri + Ric<0 

-R2c < 
-R2 + i?2c < 

Categorize the above inequalities into the following three groups: (1) group 1 not containing 
Ric, (2) group 2 containing negative Ric, (3) group 3 containing positive Ric. By adding each 
inequality from groups 2 and 3, we remove Ric. Rearranging the inequalities with respect to 
R2c, we get: 

Ri<bi+ a2 (74) 

R2 + Ri<h + a2 (75) 

-Ri < (76) 

R2c < ai (77) 

Ri + R2c < a^ (78) 

-i?2 + i?2c < (79) 

R2 - R2c < b2 (80) 

-i?2c < 0. (81) 
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Adding each inequality from groups 2 and 3, we remove i?2c and finally obtain: 

Ri < min(a5, bi + a2) (82) 

i?2 <min(65,ai + 62) (83) 

i?i + i?2 < min(65 + a2, as + 62). (84) 

Appendix C 
Converse Proof of Theorem [5] 

For completeness, we provide the detailed proof, although there are many overlaps with the 
proof in Theorem [3l The main point of the converse is how to introduce an auxiliary random 
variable U which satisfies that given Ui, Xu is conditionally independent of X2i, Claim |4l gives 
hint into this. It gives the choice ofUi := {Vl~^ , ^2'^)- 

First we consider the upper bound of an individual rate. 

where (a) follows from Fano's inequality and (6) follows from the fact that entropy is non- 
negative and conditioning reduces entropy. 
Now consider the second bound. 

NRi = H{Wi) = H{Wi\W2) 

< I{Wi; Y,''\W2) + Ncn < /(H^i; n"^, Yf\W2) + Ne^ 
^^ Y, H{Yu^ Y2^\W2^ Yr\ Yr') + Ne^ 

+ J2H{Yu\W2^Yr\Yr\Xl,Y2,,V^) + NeN 

< J2 [H{Y2,\X2^, U,) + H{Yu\Vu^ V2^^ U^)\ + Xe^ 

where (a) follows from the fact that (^1^,12^) is a function of (1^1,1^2); (&) follows from 
the fact that X^ is a function of (VI/2, ^2"^)' (<^) follows from the fact that VI is a function of 
{y^, X^\ id) follows from the fact that Vt^ is a function of ijt^ , X^^), Vt^'^^ ^ function of 
^2"^, and conditioning reduces entropy. Similarly we get the outer bound for R2. 
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The sum rate bound is given as follows. 

N{Ri + R2) = H{Wi) + H{W2) = H{Wi\W2) + H{W2) 
< I{Wi; Fi^lWs) + /(W2; ^2"^) + NeN 
= H{Y^''\W2) + I{W2] Y^) + Nei, 
= H{Y,''\W2) + H{Y2'') 

- {/7(n^, Yi'\W2) - HiY.^'lYi', W2)} + Ne^ 



HiY.'^lYf, W2) - H{Yi'\Y,'', W2) + H{Yf) + Ne 



N 



where (a) follows from the fact that X^ is a function of (1^2,^2"^) ^^^ ^1 i^ ^ function of 
(X2, 1^2 )' (b) follows from the fact that V2 is a function of X2 and conditioning reduces entropy. 
Similarly, we get the other outer bound: 

N{Ri + i?2) < 5] [H{Y2,\Vu^ V2., U,) + H{Yu)] + Ne^. 

Now let a time index Q be a random variable uniformly distributed over the set {1, 2, • • • , A^} 
and independent of {Wi, W2, Xf , Xf , Y^^ , Y^^). We define 

Xi = XiQ, Vi = 1/ig; X2 = -^2g, ^2 = ViQ, 

(85) 

n = l^ig, 1^2 = 1^2g; t/=(t/g,g). 
If (i?i,i?2) is achievable, then cat ^ as A^ ^ oc. By Claim |4l an input joint distribution 
satisfies p{u, xi,X2) = p{u)p{xi\u)p{x2\u) . This establishes the converse. 

Claim 4: Given Ui = {V^'^, ^2"^)' ^n ^^^ ^2^ are conditionally independent. 

Proof: The proof is based on the dependence-balance-bound technique in OOll , ll3T]l . For 
completeness we describe details. First we show that I{Wi] W2\Ui) = 0, which implies that Wi 
and W2 are independent given Ui. Based on this, we show that Xn and X2i are conditionally 
independent given Ui. 
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Consider 



< /(m; W2m ^^ I{Wi; W2\Ui) - /(I^i; W2) 
^^ -H{Wi) - H{W2) - H{Ui) + H{Wi, W2) 
+ H{W,, Ui) + H{W2, Ui) - H{Wi, W2, Ui) 
i -HiU,) + H{Ui\Wi) + H{Ui\W2) 

+ H{V^j,V2j\Wi,Vr\vt') 
+H{V^j,V2j\W2,Vr\vt')] 

< 

where (a) follows from I{Wi]W2) = 0; (6) follows from the chain rule; (c) follows from the 
chain rule and H{Ui\Wi, W2) = 0; (d) follows from the fact that V^ is a function of (Vl^i, V^~^) 
and V2 is a function of {W2,V(~ ) (see Claim [5]); (e) follows from the fact that conditioning 
reduces entropy. Therefore, I{Wi]W2\Ui) = 0, which shows the independence of Wi and W2 
given Ui. 

Notice that Xu is a function of {Wi, V^'^) and X2i is a function of {W2, V^~^) (see Claim 
[5]). Hence, it follows easily that 

I{Xu;X2^m = I{Xu;X2^\Vr\vr') = O, (86) 



which proves the independence of Xu and X2i given Ui. 
Claim 5: For i > 1, X^ is a function of (Vl^i, V^'^). Sir 



q-^). Similarly, X^ is a function of {W2, V^-^). 
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Proof: By symmetry, it is enough to prove it only for X{. Since the channel is deterministic 
(noiseless), X\ is a function of {Wi, W2)' In Fig. [TOl we see that information of W2 to the first 
link pair must pass through ¥21, Also note that Xu depends on the past output sequences until 
i — 1 (due to feedback delay). Therefore, X\ is a function of (l^i, V2~^). ■ 

Appendix D 
Proof of Lemma [21 

Let INR = SNR^. Then, by (29) in [[H and (77*) in (H, we get 

, /l + SNR + SNR^ + 2p*SNR^\ 
^sy. = log y i + (i_,*2)SNR- ) ' ^^'^ 

where p* is the solution between and 1 such that 

2SNR'^P*4 + SNRV' - 4(SNR'^ + SNR^)^*^ 
- (2 + SNR + 2SNR^)p* + 2(SNR^ + SNR^) = 0. 

Notice that f or < a < | and for the high SNR regime, SNR is a dominant term and < p* < L 
Hence, we get p* '^ 2SNR^~. This gives limsNR^oo iq^snr) = 1 ~ ^- For | < a < 1, the first 
and second dominant terms become SNR 2 and SNR, respectively. Also for this regime, p* ?^ L 
Hence, we approximately get 1 — p*^ ?^ SNR^^. This gives limsNR^oo iq^snr) ^ ^- ^^^ 
a > 1, note that the first and second dominant terms are SNR 2 and SNR; and p* is very close 
to 1. So we get 1 — p*^ ^ SNR 4 . This gives the desired result in the last case. 
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